test(e2e): characterize executor run-to-completion behavior#540
Open
MGudgin wants to merge 1 commit into
Open
Conversation
Member
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds cross-platform executor characterization E2E tests in wxc_e2e_tests to lock in the current “run-to-completion” behavior of the native one-shot executors (mxc-exec-mac, lxc-exec, wxc-exec.exe) ahead of the upcoming unified SandboxBackend/Runner refactor. This strengthens CI as a regression signal by turning previously-observed behavioral drift (notably on Seatbelt env/cwd inheritance) into explicit, testable contracts.
Changes:
- Add Seatbelt (macOS) executor characterization tests for exit code, stdout, env/cwd inheritance, explicit env/cwd, and timeout behavior.
- Add Bubblewrap (Linux) executor characterization tests (gated on
bwrappresence) for exit code, stdout, default env clearing, explicit env/cwd, and timeout behavior. - Add ProcessContainer (Windows) executor characterization tests gated on
MXC_E2E_HOST_PREPPED=1, plus shared helpers to locate/run the native executor and fix macOS target triple resolution.
Show a summary per file
| File | Description |
|---|---|
| src/testing/wxc_e2e_tests/tests/e2e_seatbelt_characterization.rs | Adds macOS Seatbelt run-to-completion characterization tests (including env/cwd inheritance guards). |
| src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs | Adds Linux Bubblewrap run-to-completion characterization tests, gated on bwrap availability. |
| src/testing/wxc_e2e_tests/tests/e2e_processcontainer_characterization.rs | Adds Windows ProcessContainer characterization tests gated on MXC_E2E_HOST_PREPPED=1 and host prerequisites. |
| src/testing/wxc_e2e_tests/src/lib.rs | Adds cross-platform executor helpers (has_platform_exec, run_platform_config_value, etc.) and macOS target triple support for binary discovery. |
Copilot's findings
- Files reviewed: 4/4 changed files
- Comments generated: 1
MGudgin
force-pushed
the
user/gudge/executor-characterization-tests
branch
from
89d6de2 to
8158870
Compare
Member
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
This PR adds cross-platform executor characterization tests that lock in the current run-to-completion behavior of the native one-shot executors (mxc-exec-mac / lxc-exec / wxc-exec.exe) and wires them into CI, so the upcoming unified SandboxBackend/Runner refactor can't silently change existing dev/user flows. Details * New characterization suites in wxc_e2e_tests: Seatbelt (macOS) and Bubblewrap (Linux) cover exit code, stdout, env and working-directory handling, and timeout; ProcessContainer (Windows) covers exit code/stdout/timeout and is opt-in via MXC_E2E_HOST_PREPPED=1. New harness helpers locate/run the native executor with per-run env/cwd, gate on bwrap, and fix macOS target-triple discovery. * CI wiring: the macOS job now builds/tests -p wxc_e2e_tests; the Linux job installs bubblewrap (and relaxes the Ubuntu 24.04 unprivileged user-namespace AppArmor clamp) and runs the suite. wxc_e2e_tests previously ran on neither platform. * The Seatbelt env-inheritance (empty process.env) and launcher-cwd (empty process.cwd) tests are deliberate regression guards: the refactor's always-clear-env / always-rewrite-cwd behavior will turn them red. * Not covered: stdin/TTY (the run-to-completion harness closes stdin, so the SIGTTIN regression needs a PTY harness) and Windows ProcessContainer on CI (needs a host-prepped lane). Both are follow-ups. Tests * CI green on macOS, Linux, and Windows: the macOS and Linux cargo-test jobs now execute and pass the new suites; clippy (-D warnings) and rustfmt --check are clean on aarch64-apple-darwin and x86_64-unknown-linux-gnu. * First CI execution empirically confirmed the Bubblewrap run-to-completion tree-kill gap (a forked process survives the timeout) and that Seatbelt tree-kills correctly; assertions were tuned to current behavior accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MGudgin
force-pushed
the
user/gudge/executor-characterization-tests
branch
from
90fe32e to
123d8db
Compare
MGudgin
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds cross-platform executor characterization tests that lock in the current run-to-completion behavior of the native one-shot executors (
mxc-exec-mac/lxc-exec/wxc-exec.exe) and wires them into CI, so the upcoming unifiedSandboxBackend/Runnerrefactor can't silently change existing dev/user flows.Details
wxc_e2e_tests: Seatbelt (macOS) and Bubblewrap (Linux) cover exit code, stdout, env and working-directory handling, and timeout; ProcessContainer (Windows) covers exit code/stdout/timeout and is opt-in viaMXC_E2E_HOST_PREPPED=1. New harness helpers locate/run the native executor with per-run env/cwd, gate onbwrap, and fix macOS target-triple discovery.-p wxc_e2e_tests; the Linux job installsbubblewrap(and relaxes the Ubuntu 24.04 unprivileged user-namespace AppArmor clamp) and runs the suite.wxc_e2e_testspreviously ran on neither platform.process.env) and launcher-cwd (emptyprocess.cwd) tests are deliberate regression guards: the refactor's always-clear-env / always-rewrite-cwd behavior will turn them red.Tests
clippy(-D warnings) andrustfmt --checkare clean onaarch64-apple-darwinandx86_64-unknown-linux-gnu.